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ABSTRACT 



A study was conducted to determine the effectiveness of 
using advance organizers in instruction at the graduate level. Participants 
were 218 graduate students in research methodology courses. Fifty-four 
students were enrolled in sections in which advance and post organizers were 
used; 164 were enrolled in sections in which this type of instruction did not 
take place. Conceptual knowledge, involving students' command of research 
concepts, methodologies, and applications, was measured individually in both 
sets of classes through midterm and final examinations. Findings show that 
students in the advance organizer sections of the course obtained higher 
levels of overall achievement than did their counterparts. The effect size 
pertaining to this difference was 0.54, which is considered moderate. The 
implications of these findings are discussed, as are recommendations for 
incorporating advance and post organizers in research methodology courses. 
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Abstract 

In an effort to increase achievement levels in graduate -level research 
methodology courses, some instructors are using advance and post organizers. 
However, to date, no study has investigated the effectiveness of this 
instructional technique in these courses. This was the purpose of the present 
study. Although a myriad of studies has been conducted during the past three 
decades assessing the effectiveness of advance organizers, few of these 
investigations have been undertaken at the graduate level. Subjects comprised 
218 students, enrolled in a mid-southern university's graduate-level research 
methodology courses. Fifty-four students were enrolled in sections in which 
advance and post organizers were used; 164 were enrolled in sections in which 
this method of instruction did not take place. Conceptual knowledge, involving 
students' command of research concepts, methodologies, and applications, was 
measured individually in both sets of classes via midterm and final examinations. 
Findings revealed that students enrolled in the advance organizer sections of 
the course obtained higher levels of overall achievement than did their 
counterparts. The effect size pertaining to this difference was .54, which is 
considered moderate. The implications of these findings are discussed, as are 
recommendations for incorporating advance and post organizers in research 
methodology courses . 
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The Effect of Advance Organizers on Achievement 
in Graduate-Level Research Methodology Courses 

As the importance of research is being recognized increasingly, so more 
students are required to enroll in research methodology courses as a necessary 
part of their graduate degree programs . Unfortunately, it appears that these 
courses are exceedingly difficult for many students (Onwuegbuzie , 1997; 
Onwuegbuzie, Slate, Paterson, Watson, & Schwartz, 1998) . Indeed, these students 
typically experience lower levels of performance in these courses than in their 
other graduate-level classes (Onwuegbuzie et al., 1998). In an attempt to 
increase achievement levels, research methodology instructors have begun to 
experiment with a variety of teaching strategies. However, many of these 
strategies, such as cooperative learning (Onwuegbuzie & DaRos , 1998), have led 
to mixed findings with respect to their effectiveness. 

Since many students report that research methodology courses are far 
removed from their own fields, they have difficulty adjusting their cognitive set 
to the study of this discipline (Onwuegbuzie, 1997) . Thus, although not yet 
tested empirically, it is possible that the subsumption theory of learning, which 
is based on the assumption that cognitive structure is hierarchically organized, 
may be particularly pertinent in research methodology classes . According to this 
theory, which was first introduced by Ausubel (1960) , meaningful learning occurs 
when an individual's existing knowledge interacts with new information in a non- 
arbitrary way. Relevant prior knowledge, in the form of general ideas, 
facilitates the incorporation of meaningful concepts into this cognitive 
structure. Ausubel maintains that learning a new concept is meaningful when it 
can be classified correctly and arranged in the learner's memory and can, 
therefore, be retained for a long time. Accordingly, new specific information 
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is learned and stored in hierarchical form, with broader and more abstract 
previously learned concepts subsuming more specific, new concepts. Ausubel thus 
predicted that advance organizers enable students to encode, to organize, to 
retain, and to recall any subsequently read material to be learned. 

The advance organizer is a set of instructional materials which is related 
to new material that is presented on a higher level of abstraction, 
inclusiveness, and generality than the more detailed and differentiated material 
to be learned (Ausubel, 1968) . The overall goal of the advance organizer, which 
is presented to students before they read unfamiliar material, is to bridge the 
gap between what learners already know and what they need to know for 
comprehension of a new concept to occur. According to Ausubel (1968), advance 
organizers help "to provide ideational scaffolding for the stable incorporation 
and retention of the more detailed and differentiated material that follows" (p. 
148) . Advance organizers are different from summaries or overviews which are 
presented at the same level of abstraction, inclusiveness, and generality, and 
which only emphasize important points, thus achieving their effects by repetition 
and selective emphasis on key words and central concepts (Ausubel, 1963) . 

Ausubel (1963) theorized that three of the main factors which influence the 
learning and retention of meaningful information are (1) the availability of 
relevant anchoring ideas, (2) the stability (i.e., clarity and organization) of 
these ideas, and (3) the distinguishability of the new ideas from the anchoring 
ideas. By recalling frameworks of previously learned concepts, advance organizers 
can be used to provide ideational scaffolding for new ideas and to discern 
similarities and differences between the new material and previously learned 
concepts . 

Similarly, advance organizers have been postulated as facilitating the 
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incorporation and learning of new material in the following three ways: (1) by 
activating relevant subsumers already present in the learner's cognitive 
structure, (2) by providing subsumers where none previously exist, and (3) by 
reducing the importance of rote memorization, since key anchors are supplied to 
which new material can be meaningfully related (Lawton, 1977) . 

Advance organizers also have been conceptualized as falling into one of two 
categories: comparative and expository (Mayer, 1979). Comparative organizers 
activate already existing schemata and thereby allow for enhanced learning of 
subsequent content (Ausubel, 1968) . In contrast, expository organizers provide 
knowledge which readers have not yet acquired in order to allow for the 
comprehension of subsequent material (Ausubel, 1968) . 

Advance organizers can be presented in many forms. For example, the 
instructor could provide an introductory overview in which the entire lesson or 
course is presented in outline form. According to Botwinick (1978) , this 
provides an early opportunity for the learner to see the "map" which is being 
followed. Additionally, the advance organizer could take the form of an 
introductory statement about the content of a lesson or a series of readings 
followed by several key questions for learners to consider while undertaking the 
assigned readings or participating in the lesson. Peterson (1983) contends that 
these questions can help the learners to develop a purpose for the readings and 
to organize the material as they read or listen. Advance organizers also could 
take the form of a set of notes to follow or a list of concepts or facts to be 
examined . . 

Increasingly, advance organizers have been presented graphically. These 
graphic organizers now are referred to as concept maps. These concept maps are 
"diagrammatic representations of the basic vocabulary of a unit so as to show 
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relationships among the concepts represented by those words" (Herber & Sanders, 
1969, p. 4) . Concept maps have been recommended for use in chemistry (Novak, 
1984) , physics and literature (Moreira, 1985) , reading (Gold, 1984) , social 
studies (Wease, 1986) , and computer-assisted instruction (Heinze-Fry, Crovello, 
& Novak, 1984) . Indeed, some researchers contend that, given proper instruction 
in concept mapping, a student of any ability level can construct a concept map. 

Despite some inconsistency in the research findings (Anderson, Spiro, & 
Anderson, 1978; Barnes & Clawson, 1975; Clark & Bean, 1980; Hartley & Davis, 
1976) , many researchers who have utilized meta-analytic techniques have reported 
that advance organizers exert a modest but significant effect in facilitating 
learning (Kozlow, 1978; Luiten, Ames, & Ackerson, 1980; Mayer, 1979; Moore & 
Readance, 1984) . In other meta-analysis reviews, Walberg (1984) found an effect 
size of 0.20, and Stone (1982) reported an effect size of .48. In fact, Luiten 
et al . (1980) concluded that the benefit of advance organizers extended to "all 
content areas examined ... and with individuals of all grade and ability levels" 
(p. 217) . Moreover, the effects of advance organizers tend to be optimal if they 
are well learned, are presented in a concrete rather than abstract manner, and 
are followed by a moderate delay prior to reading the to-be- learned material 
(Corkill, Glover, Bruning, & Krug, 1988; Dinnel & Glover, 1985; Mar shark, 1985; 
Mayer, 1984). In addition, some researchers (e.g., Lenz , Alley, & Schumaker, 
1987) contend that advance organizers are effective only when students are taught 
to use such techniques. 

Advance organizers have been. found to promote retention of conceptual but 
not factual information (Mayer & Bromage, 1980) and to enhance problem solving 
which involves transfer (Mayer, 1980) . According to Mayer (1979, 1980) , advance 
organizers are even more beneficial for inexperienced or low-ability learners 
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than for higher ability learners. In any case, this form of presentation appears 
to be useful whether it is utilized before (i.e., advance organizers) or after 
(i.e., post organizers) the text (Luiten et al., 1980). Some researchers contend 
that advance organizers can be particularly beneficial for older students. In 
particular, Peterson (1983) asserts that advance organizers can bridge the gap 
between what older students already know and what is being learned. 

With respect to concept maps, Novak, Gowin, and Johansen (1983) found that 
students who were trained in concept and vee mapping tended to exhibit knowledge 
of more concept relationships than did students who were not trained. 
Additionally, Willerman and Mac Harg (1991) reported an effect size of .40 for 
eighth-grade students enrolled in science classes, in favor of those who used 
concept maps. Indeed, it has been found that even more information is recalled 
when graphic elements are incorporated into the advance organizer (Hawk, McLeod, 
Sc Jeane, 1981; Schwartz & Kulhavy, 1982; Winn, 1982). 

Surprisingly, despite the fact that a myriad of studies has been conducted 
during the past four decades assessing the effectiveness of advance organizers, 
few of these investigations have been undertaken at the graduate level. Thus, 
the authors sought to investigate whether achievement in graduate -level research 
methodology classes increases when students are introduced to advance organizers. 
Since most concepts which are introduced in research methodology courses form 
part of a system of ideas, and since these concepts tend to be related inherently 
to other ideas in the system which already are understood by the learner, it was 
hypothesized that graduate students enrolled in research methodology courses in 
which advance organizers are used would have higher levels of achievement than 
their counterparts who are not taught using advance organizers. 
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Method 

Subjects and Procedure 

Subjects comprised 218 students, who were enrolled in several sections of 
a graduate -level research methodology course over a two-year period. Fifty-four 
students were enrolled in the three sections in which advance organizers were 
used, whereas the remaining 164 students were enrolled in the eight sections in 
which advance organizers were not utilized. The same instructor taught all 
sections. The major course requirements, which were group-based, consisted of 
written critical evaluations of published research reports and preparation of 
research proposals. In-class examinations, comprising a midterm and a final, 
were administered individually. 

With respect to the advance organizer groups, the majority of participants 
was female (80.2%), ranging in age from 23 to 56 (M=32.3, SD = 7.8), and with 
a mean grade point average of 3.67 (SD = 0.41) . With regard to the control group 
sample, most of the participants were female (79.3%), ranging in age from 22 to 
55 (M = 31.7, SD = 8.4), and with a mean grade point average of 3.65 (SD = 0.40) . 
No significance difference (p < .05) in mean grade point average was found 
between students in the advance organizer classes and those in the traditional 
classes . 

According to the university graduate handbook, the course involved the 
"application of scientific method to educational research, including nature of 
research problems in education, theory of research, experimental design, 
techniques in data gathering, the interpretation of results, research reporting, 
and bibliographical techniques." For each 16-week semester, classes were held 
for three hours, once per week. The main requirement of the course was the 
completion of a research proposal. The objective of the proposal was to prepare 
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students throughly to be able to write proposals for dissertations and for 
seeking external funding. As such, the research proposals provided authentic 
assessment. Specifically, the research proposal, which could represent either 
quantitative or qualitative research on a topic of the student's choice, had to 
comprise a title, introduction section, review of the related literature, 
methodology section, analysis section, bibliography, and appendix section 
including a biography of the proposal writer, a timetable and budget, consent 
form(s) , and author-designed instrument (s) . Research proposals had to be unique, 
realistic, have educational significance, and extend the knowledge base. 
Students were expected to type their proposals, following guidelines specified 
by the American Psychological Association (1994) . Students' writing style (e.g., 
grammar, punctuation, clarity, and application of American Psychological 
Association criteria (1994)) also was assessed. All proposals had to include an 
in-depth review of the literature, and thus extensive library usage was required. 
Indeed, although many research methodology instructors appear to require what 
could be conceptualized as a mini -proposal , the research proposal in this course 
was required to be extremely comprehensive. Historically, over the years, 
research proposals in this course typically have ranged from 25 to 40 pages, with 
the literature review section usually ranging from 5 to 15 pages. Students in 
both treatment conditions were encouraged to immerse themselves in their research 
proposals from the first class meeting. Moreover ,. students/groups were required 
to formulate their research questions by the second class meeting and to start 
obtaining literature sources by the third class meeting. 

The second major course requirement involved a written critical evaluation 
of a published research report (article critique) . The major goal of the article 
critique was to allow students to practice evaluating published research articles 




Advance Organizers 



10 



utilizing principles of the scientific method. In order to prevent students from 
procrastinating, students were required to select several potential articles to 
critique and to bring them to the second class meeting for advice from the 
instructor as to their appropriateness. Furthermore, students were required to 
make their final selection as to which article to critique by the third week of 
the semester. 

In both conditions, on the first day of class, students, in turn, were 
asked to introduce themselves to the whole class, disclosing their major, 
educational aspirations, profession, and interests. Following these 
introductions, students were asked to form groups comprising 3-4 individuals. 
Students were encouraged to choose group members based on major, profession, and 
proximity to each other's homes. The cooperative learning group that was utilized 
involved the use of base groups (Smith, Johnson, & Johnson, 1992) . The aim of 
these base groups was to promote stable membership whose foremost responsibility 
was to provide each student the support, encouragement, and assistance needed to 
understand the material presented by the instructor and in the readings, with a 
view to (1) completing the group assignments successfully and (2) preparing 
students for the in-class examinations. Students were encouraged to stay 
together during the entire course. Although they were allowed to change groups 
if any conflicts or unresolvable problems arose among group members, no student 
requested such a change. Students were asked to exchange phone numbers and e- 
mail addresses and information about their schedules so that they could meet 
outside class. Each base group undertook one research proposal and one article 
critique . 

The instructor informed students of the following basic group skills: every 
group member should participate as equally as possible, or at least according to 
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their strengths, students should respect the opinions of all group members, no 
students should dominate group discussions, and every student should be aware of 
all tasks undertaken by group members and to be prepared to provide constructive 
criticism. Students were not assigned specific group roles; however, they were 
presented with different models for the division of labor (e.g., each student 
writing a section of the research proposal and article critique; each student 
individually undertaking all sections of these assignments and then comparing 
their work with all other group members with a view to merging) . 

The only difference between the experimental and control groups was that 
advance and post organizers were utilized with the former. In the advance 
organizer conditions, a conceptually-oriented advance organizer was prepared for 
each chapter of the textbook. In addition, in each class period, the instructor 
allocated time for each cooperative base group to construct concept maps (i.e., 
post organizers) pertaining to each unit or chapter read previously. While 
students worked in groups, the instructor observed, answered questions posed by 
students, and informed the class of any insights gained from circulating among 
the groups. After a specified period of time, each base group would then display 
their concept maps to the other groups, who would subsequently critique them and 
compare and contrast these maps with their own graphical displays. From these 
between-group discussions, a consensus would then be reached as to which elements 
of the concept maps were the most appropriate . At this point , the instructor 
would check the whole-class concept map which emerged for accuracy and 
completeness. An example of a concept map which emerged is displayed in Figure 
1 . 

Instruments 

A scoring rubric was used to evaluate proposals and article critiques, with 
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detailed feedback provided. Students in both groups were given group scores for 
these assignments. Conceptual knowledge, which involved students' knowledge of 
research concepts, methodologies, and applications, was measured individually in 
both sets of classes via comprehensive written midterm and final examinations. 

Results 

An independent t-test revealed that students enrolled in the advance 
organizer sections of the course obtained higher (t = 4.9, p < .0001) levels of 
overall achievement (M = 87.8%, SD = 4.7%) than did their counterparts (M = 
83.3%, SD = 8.4%) . The effect size pertaining to this difference was .54, which 
is considered moderate (Cohen, 1988) . 

Discussion 

Failure to understand research concepts may stem from inadequate 
utilization of existing prior knowledge (Spiro & Tirre, 1980) . Since advance 
organizers are designed to activate relevant subsumers already present in the 
learner's cognitive structure (Ausubel, 1960, 1963, 1968), it was hypothesized 
that graduate students enrolled in research methodology courses in which advance 
organizers are used would have higher levels of achievement than their 
counterparts who are not taught using advance organizers. The results of the 
present study clearly support the effectiveness of advance organizers in 
promoting learning in research methodology courses, although caution should be 
exercised in interpreting the findings, due to the quasi -experimental nature of 
the study. 

It is likely that advance organizers may have been effective in research 
methodology courses because they provided students with more direction for 
learning the concepts and facts which were presented by the instructor (Willerman 
Sc Mac Harg, 1991) . That is, the advance organizers cued students as to what to 
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look for as they read the written materials. In particular, advance organizers 
may have helped students to examine cause and effect, to compare and to contrast 
concepts, and to investigate sequences of events and other relationships {Hawk, 
1986) . The advance organizers, especially the concept maps, may have been 
successfully used by many students as review instruments, strengthening their 
retention by giving stability to new information as a result of their subsuming 
qualities (Hawk, 1986) . The concepts maps also may likely have provided 
reference points which assisted learners in assimilating new material and 
organizing the main concepts into a logical framework (Ausubel, 1963) . This, in 
turn, may have activated many students' schemata relevant to the to-be-learned 
material. It is also possible that the advance organizers provided a scaffold 
or framework which was used to categorize, to encode, to store, and to recall new 
concepts (Ausubel, 1963) . 

Many graduate students, even those with high levels of overall academic 
achievement, tend to employ inappropriate study habits when attempting to learn 
research and statistical concepts (Onwuegbuzie , DaRos, & Ryan, 1997) . In 
particular, these students often read to-be-learned material too quickly, which 
threatens their ability to understand fully many concepts (Onwuegbuzie et al. 
1997) . Thus, teaching students to utilize advance and post organizers may help 
to prevent them from reading material too quickly, as well as motivate them to 
modify material in light of material read in previous pages. 

When students are required to learn new and unfamiliar research concepts 
without the benefit of organizers which subsume these concepts, the learning 
which occurs is most likely to be rote because students perceive that they have 
to memorize the material verbatim (Alexander, Frankiewicz, & Williams, 1979) . 
Thus, in the present study, advance organizers may have increased performance 
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levels by helping to minimize rote learning and, at the same time, promoting 
meaningful learning and retention of research concepts. 

Replications of this study are needed using an experimental design in order 
to assess the internal validity of the present findings. Nevertheless, the 
current results suggest that advance organizers can be an important instructional 
tool in research methodology courses, with the potential to increase achievement 
levels. Further research is needed to clarify the relationship between advance 
organizers and student performance in research methodology courses and to 
identify which types of advance organizers maximize meaningful learning in these 
classes . 
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Figure Caption 

Figure 1 . Example of Student-Generated Concept Map 
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